Fluent bit回顧
Log Agent - Fluent Bit 簡介
Log Agent - Fluent Bit 安裝與常見架構模式

Service Configuration

Fluent bit的配置起點都預設先讀取fluent-bit.conf這檔案
這配置通常, 第一個Section是Service, 依序是Input、Parser、Filter、Output

配置方式很像INI
都有Section、Key Value, 透過Indent(4spaces)做縮排, 使用#做行註解

[SERVICE]
    # comment, 註解也必須做indent縮排
    daemon    off
    log_level debug
允許空行, 但空行別做indent, 不然會被認為是key
[INPUT]
    name forward
    listen 0.0.0.0
    port 24224

Service Section Properties

• Flush

  DataPipe的圖中有的Buffer,
  這值用來描述, Buffer內的資料每隔幾秒批量寫到Output元件內
  預設值是5秒

• Daemon

  是否以daemon後台方式運作
  以Container形式的話就隨意了XD
  預設是off

• Log_FIle

  Fluent bit服務的log輸出到指定的路徑
  預設是都輸出到stdout

• Log_Level

  跟很多application一樣, 也能設定什麼等級的log才做輸出
  有error、warning、info、debug、trace
  預設是info

• Parsers_File

  指定parsers配置檔案的位子
  這配置可以出現多次, 如果你有多個parse.cnf需要被引入的話

[SERVICE]
    # comment, 註解也必須做indent縮排
    daemon    off
    log_level debug
    Parsers_File parsers.conf
    Parsers_File custom_parsers.conf

• Plugins_File

  跟Parsers_File一樣, 用來引入plugins

• Health_Check

  啟動health_check, 需要跟http_server搭配
  預設是off

• HC_Errors_Count

  fluent bit若連續5次error, 才判定為unhealthy
  預設是5次

• HC_Retry_Failure_Count

  retry failure的次數
  預設是5次

• HC_Period

  每次retry的時間間隔
  預設60s
  

• HTTP_Server

  是否啟用fluent bit內建的http server
  預設是off

• HTTP_Listen

  啟動監聽的port
  預設是本機的所有網路接口 0.0.0.0

• HTTP_Port

  監聽請求的TCP port
  預設是2020
  通常開啟http設定, 是用在fluent bit本身要給人或其他監控服務給監控, 才會特地開

內建REST API

來玩一下fluent bit 提供的REST api

version: '3'
services:
  fluentd:
    image: fluent/fluent-bit
    volumes:
        - ./conf:/fluent-bit/etc
    ports:
      - "24224:24224"
      - "24224:24224/udp"
      - "2020:2020" # mapping http port 
    hostname: fluentd
    logging:
      driver: "json-file"
      options:
          max-size: 100m
          max-file: "5"

[SERVICE]
    flush 1
    Daemon off
    log_level error
    #開啟http server
    http_server on
    #監聽所有網卡
    http_listen 0.0.0.0
    #監聽導到port 2020的請求
    http_port 2020
    storage.metrics on

啟動容器
docker-compose up -d
然後先安裝一下Day15提到的JQ套件
因為它回應是JSON, JQ讓Response好閱讀

• 顯示所有fluent bit的資訊

# 可以看到版本是v1.8.7, 然後是community版
curl -s http://127.0.0.1:2020  | jq              
{
  "fluent-bit": {
    "version": "1.8.7",
    "edition": "Community",
    "flags": [
      "FLB_HAVE_PARSER",
      "FLB_HAVE_RECORD_ACCESSOR",
      "FLB_HAVE_STREAM_PROCESSOR",
      "FLB_HAVE_TLS",
      "FLB_HAVE_OPENSSL",
      "FLB_HAVE_METRICS",
      "FLB_HAVE_AWS",
      "FLB_HAVE_AWS_CREDENTIAL_PROCESS",
      "FLB_HAVE_SIGNV4",
      "FLB_HAVE_SQLDB",
      "FLB_HAVE_METRICS",
      "FLB_HAVE_HTTP_SERVER",
      "FLB_HAVE_SYSTEMD",
      "FLB_HAVE_FORK",
      "FLB_HAVE_TIMESPEC_GET",
      "FLB_HAVE_GMTOFF",
      "FLB_HAVE_UNIX_SOCKET",
      "FLB_HAVE_PROXY_GO",
      "FLB_HAVE_JEMALLOC",
      "FLB_HAVE_LIBBACKTRACE",
      "FLB_HAVE_REGEX",
      "FLB_HAVE_UTF8_ENCODER",
      "FLB_HAVE_LUAJIT",
      "FLB_HAVE_C_TLS",
      "FLB_HAVE_ACCEPT4",
      "FLB_HAVE_INOTIFY"
    ]
  }
}

• 顯示服務已經啟動了多久

curl -s http://127.0.0.1:2020/api/v1/uptime | jq 
{
  "uptime_sec": 962,
  "uptime_hr": "Fluent Bit has been running:  0 day, 0 hour, 16 minutes and 2 seconds"
}

• 顯示服務的各項metrics
  因為這裡我還是用上一篇的配置, 所以它input就顯示forward的一些資訊

curl -s http://127.0.0.1:2020/api/v1/metrics/ | jq          
{
  "input": {
    "forward.0": {
      "records": 52,
      "bytes": 10548
    }
  },
  "filter": {
    "record_modifier.0": {
      "drop_records": 0,
      "add_records": 0
    }
  },
  "output": {
    "stdout.0": {
      "proc_records": 52,
      "proc_bytes": 11692,
      "errors": 0,
      "retries": 0,
      "retries_failed": 0,
      "dropped_records": 0,
      "retried_records": 0
    }
  }
}

• 顯示服務的各項metrics, 但以Prometheus server的格式
  這裡就不是用JSON格式輸出了, 而是Prometheus Text

curl -s http://127.0.0.1:2020/api/v1/metrics/prometheus  
# HELP fluentbit_filter_add_records_total Fluentbit metrics.
# TYPE fluentbit_filter_add_records_total counter
fluentbit_filter_add_records_total{name="record_modifier.0"} 0 1633928614748
# HELP fluentbit_filter_drop_records_total Fluentbit metrics.
# TYPE fluentbit_filter_drop_records_total counter
fluentbit_filter_drop_records_total{name="record_modifier.0"} 0 1633928614748
# HELP fluentbit_input_bytes_total Number of input bytes.
# TYPE fluentbit_input_bytes_total counter
fluentbit_input_bytes_total{name="forward.0"} 10548 1633928614748
# HELP fluentbit_input_records_total Number of input records.
# TYPE fluentbit_input_records_total counter
fluentbit_input_records_total{name="forward.0"} 52 1633928614748
# HELP fluentbit_output_dropped_records_total Number of dropped records.
# TYPE fluentbit_output_dropped_records_total counter
fluentbit_output_dropped_records_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_errors_total Number of output errors.
# TYPE fluentbit_output_errors_total counter
fluentbit_output_errors_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_proc_bytes_total Number of processed output bytes.
# TYPE fluentbit_output_proc_bytes_total counter
fluentbit_output_proc_bytes_total{name="stdout.0"} 11692 1633928614748
# HELP fluentbit_output_proc_records_total Number of processed output records.
# TYPE fluentbit_output_proc_records_total counter
fluentbit_output_proc_records_total{name="stdout.0"} 52 1633928614748
# HELP fluentbit_output_retried_records_total Number of retried records.
# TYPE fluentbit_output_retried_records_total counter
fluentbit_output_retried_records_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_retries_failed_total Number of abandoned batches because the maximum number of re-tries was reached.
# TYPE fluentbit_output_retries_failed_total counter
fluentbit_output_retries_failed_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_retries_total Number of output retries.
# TYPE fluentbit_output_retries_total counter
fluentbit_output_retries_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_uptime Number of seconds that Fluent Bit has been running.
# TYPE fluentbit_uptime counter
fluentbit_uptime 1164
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1633927450
# HELP fluentbit_build_info Build version information.
# TYPE fluentbit_build_info gauge
fluentbit_build_info{version="1.8.7",edition="Community"} 1

• Health check
  如果服務健康, 則Status是200, body是OK
  反之則是500, body是error

curl -s -I http://127.0.0.1:2020/api/v1/health
HTTP/1.1 200 OK
Server: Monkey/1.7.0
Date: Mon, 11 Oct 2021 05:05:47 GMT
Transfer-Encoding: chunked

---參考文件
Configureation File
Monitoring